An apparatus for providing semantic information and a method of operating the same

ABSTRACT

According to an aspect, there is provided an apparatus (100) comprising a display surface (102) for presenting an image of a user; a light emitting portion (104) for using light to convey semantic information to the user; a sensor (108) for measuring the position of the user relative to the light emitting portion (104); and a control unit (106) configured to control the light emitting portion (104) based on the measured position of the user relative to the light emitting portion (104) so as to minimise the effect of the light conveying the semantic information provided by the light emitting portion (104) on the presented image of the user.

FIELD OF THE INVENTION

The invention relates to an apparatus that comprises a display surface,such as a reflective surface/mirror or a display screen, and that canuse light to provide semantic information to a user of the apparatus.

BACKGROUND OF THE INVENTION

During personal care activities a mirror is often used. Such mirrorsshould provide the user with good visibility of the face and/or body,e.g. to support the application of make-up to the face, carrying outskin care activities, grooming (e.g. shaving or hair trimming), oralcare, applying or removing contact lenses, etc.

To improve the lighting conditions and thus improve the visibility ofthe user's face and/or body in the mirror, one or more light sources areoften used to illuminate the user. These light source(s) can beintegrated with the mirror (for example located in or on one or moreedges of the mirror), or they can be separate from the mirror.

As an alternative to traditional mirrors that include a reflectivesurface, so-called ‘virtual mirrors’ or ‘digital mirrors’ have beendeveloped that use one or more cameras or other image or video capturedevices to capture images or video of the user and present them on adisplay screen in real-time (or near-real time). These virtual mirrorscan also have an integrated or separate lighting system for improvingthe lighting conditions and thus improve the displayed image of theuser.

SUMMARY OF THE INVENTION

To improve the performance of personal care activities, it has beensuggested that mirrors (whether reflective surface-based or virtual) canbe used to present information to the user about the personal careactivity while the user is using the mirror. Similarly, while a user isusing a mirror, the mirror can be used to present other relevantinformation to the user, for example the news headlines or a weatherreport.

For some semantic information (e.g. information relating to theperformance of the personal care activities or other information that isuseful to the user) it is possible to provide the information to theuser using a display integrated into the mirror. However, this is notalways desirable as the displayed information can interfere with (e.g.overlay) the reflected image of the user. Hence, using an integrateddisplay inside the mirror is not always an ideal solution. Instead, itis proposed to present ‘semantic information’ to the user using alighting system that is integrated or used with mirrors, or a separatelighting system that is provided in, on or with the mirror for thespecific purpose of providing semantic information.

However, it is also desirable for the user to be evenly lit (either withthe ambient light or by the integrated or separate lighting system) sothat the user can optimally see their face and/or body in the mirrorduring a personal care activity, and using light to present semanticinformation to the user can affect the evenness of the lighting on theuser (and thus on the image of the user presented by the mirror). Forexample, red light can be used to signify to the user that they have notbrushed their teeth for a sufficient amount of time, but this will alsocast red light onto the user's face and/or create shadows on the user'sface, which will be visible to the user in the mirror.

Thus, there is a need for an improved apparatus and method of operatingan apparatus that minimises the effect of light conveying semanticinformation on a presented image of a user.

According to a first aspect, there is provided an apparatus comprising adisplay surface for presenting an image of a user; a light emittingportion for using light to convey semantic information to the user; asensor for measuring the position of the user relative to the lightemitting portion; and a control unit configured to control the lightemitting portion based on the measured position of the user relative tothe light emitting portion so as to minimise the effect of the lightconveying the semantic information provided by the light emittingportion on the presented image of the user.

In some embodiments, the display surface can present the image of theuser in real-time. It will be appreciated that the real-time image maybe a dynamic, moving image, otherwise referred to as a live image of theuser.

In some embodiments the effect that is minimised is a shadowing and/orcolour casting effect of the light conveying the semantic informationprovided by the light emitting portion on the user and the presentedimage of the user.

In some embodiments, the display surface is a mirror or other reflectivesurface. In other embodiments, the display surface is a display screenthat displays images or video captured or obtained by one or morecameras or other image or video capture devices. In these embodiments,the apparatus can be a virtual mirror, a digital mirror or a smartmirror.

In some embodiments, the control unit is configured to control the lightemitting portion by adjusting the brightness or intensity of the lightused to convey the semantic information based on the measured positionof the user relative to the light emitting portion. In some embodiments,the control unit is configured to control the brightness or intensity ofthe light emitting portion such that the brightness or intensity of thelight conveying the semantic information is lower when the user iscloser to the light emitting portion than when the user is further fromthe light emitting portion.

In some embodiments, the control unit is configured to control the lightemitting portion by adjusting the colour, colour saturation and/orcolour blend of the light used to provide the semantic information basedon the measured position of the user relative to the light emittingportion.

In some embodiments, the light emitting portion comprises a plurality oflight emitting elements that can be used to convey the semanticinformation, and the control unit is configured to control the number oflight emitting elements used to convey the semantic information based onthe measured position of the user relative to the light emittingportion. The control unit can be configured to control the number oflight emitting elements used to convey the semantic information suchthat the number of light emitting elements used to convey the semanticinformation is lower when the user is closer to the light emittingportion than when the user is further from the light emitting portion.

In some embodiments, the light emitting portion comprises a plurality oflight emitting elements that can be used to convey the semanticinformation, and the control unit is configured to control a lightemission angle or focus of the light emitted by the light emittingelements based on the measured position of the user relative to thelight emitting portion.

In some embodiments, the sensor comprises a distance sensor, a proximitysensor, a camera or image sensor.

In some embodiments, the apparatus further comprises a sensor forobtaining one or more images or a video sequence of a user of theapparatus; wherein the control unit is further configured to analyse theobtained one or more images or video sequence to determine an effect ofthe light conveying the semantic information on the user and/or todetermine the evenness of lighting on the user; and control the lightemitting portion based on the measured position of the user relative tothe light emitting portion and the determined effect of the light and/ordetermined evenness of the lighting so as to minimise the effect of thelight conveying the semantic information provided by the light emittingportion on the presented image of the user. This embodiment has anadvantage that it can adjust the light emitting portion to take intoaccount the actual effect of the light conveying the semanticinformation on the user.

In some embodiments, the apparatus further comprises a sensor fordetecting an ambient light level or contribution of ambient light; andthe control unit is configured to control the light emitting portion soas to minimise the effect of the semantic information provided by thelight emitting portion on the presented image of the user based on theambient light and the measured position of the user relative to thelight emitting portion. In these embodiments, the sensor for detectingan ambient light level or contribution of ambient light can be a lightsensor, a camera or an image sensor. This embodiment has an advantagethat the ambient light level is taken into account when adjusting thelight emitting portion in order to effect an appropriate level ofadjustment to the light conveying the semantic information.

In some embodiments, the display surface is a reflective surface, e.g.as in a conventional mirror. In other embodiments, the display surfaceis a display screen that displays an image or video sequence obtainedusing a camera or an image sensor, i.e. a virtual mirror.

According to a second aspect, there is provided a method of operating anapparatus to convey semantic information to a user of the apparatus, theapparatus comprising a display surface for displaying an image of theuser, the method comprising measuring the position of a user relative toa light emitting portion of the apparatus; and controlling the lightemitting portion to emit light to convey the semantic information to theuser, wherein the light emitting portion is controlled based on themeasured position so as to minimise the effect of the light conveyingthe semantic information on an image of the user presented on thedisplay surface.

In some embodiments, the display surface can present the image of theuser in real-time. It will be appreciated that the real-time image maybe a dynamic, moving image, otherwise referred to as a live image of theuser.

In some embodiments the effect that is minimised is a shadowing and/orcolour casting effect of the light conveying the semantic informationprovided by the light emitting portion on the user and the presentedimage of the user.

In some embodiments, the display surface is a mirror or other reflectivesurface. In other embodiments, the display surface is a display screenthat displays images or video captured or obtained by one or morecameras or other image or video capture devices. In these embodiments,the apparatus can be a virtual mirror, a digital mirror or a smartmirror.

In some embodiments, the step of controlling comprises controlling thelight emitting portion by adjusting the brightness or intensity of thelight used to convey the semantic information based on the measuredposition of the user relative to the light emitting portion. In someembodiments, the step of controlling comprises controlling thebrightness or intensity of the light emitting portion such that thebrightness or intensity of the light conveying the semantic informationis lower when the user is closer to the light emitting portion than whenthe user is further from the light emitting portion.

In some embodiments, the step of controlling comprises controlling thelight emitting portion by adjusting the colour, colour saturation and/orcolour blend of the light used to provide the semantic information basedon the measured position of the user relative to the light emittingportion.

In some embodiments, the light emitting portion comprises a plurality oflight emitting elements that can be used to convey the semanticinformation, and the step of controlling comprises controlling thenumber of light emitting elements used to convey the semanticinformation based on the measured position of the user relative to thelight emitting portion. The step of controlling can comprise controllingthe number of light emitting elements used to convey the semanticinformation such that the number of light emitting elements used toconvey the semantic information is lower when the user is closer to thelight emitting portion than when the user is further from the lightemitting portion.

In some embodiments, the light emitting portion comprises a plurality oflight emitting elements that can be used to convey the semanticinformation, and the step of controlling comprises controlling a lightemission angle or focus of the light emitted by the light emittingelements based on the measured position of the user relative to thelight emitting portion.

In some embodiments, the step of measuring the position of the userrelative to the light emitting portion of the apparatus comprisesmeasuring the position using a distance sensor, a proximity sensor, acamera or image sensor.

In some embodiments, the method further comprises the steps of obtainingone or more images or a video sequence of a user of the apparatus;analysing the obtained one or more images or video sequence to determinean effect of the light conveying the semantic information on the userand/or to determine the evenness of lighting on the user; and the stepof controlling comprises controlling the light emitting portion based onthe measured position of the user relative to the light emitting portionand the determined effect of the light and/or determined evenness of thelighting so as to minimise the effect of the light conveying thesemantic information provided by the light emitting portion on thepresented image of the user. This embodiment has an advantage that itcan adjust the light emitting portion to take into account the actualeffect of the light conveying the semantic information on the user.

In some embodiments, the method further comprises the step of detectingan ambient light level or contribution of ambient light; and the step ofcontrolling comprises controlling the light emitting portion so as tominimise the effect of the semantic information provided by the lightemitting portion on the presented image of the user based on the ambientlight and the measured position of the user relative to the lightemitting portion. In these embodiments, the sensor for detecting anambient light level or contribution of ambient light can be a lightsensor, a camera or an image sensor. This embodiment has an advantagethat the ambient light level is taken into account when adjusting thelight emitting portion in order to effect an appropriate level ofadjustment to the light conveying the semantic information.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the invention, and to show more clearlyhow it may be carried into effect, reference will now be made, by way ofexample only, to the accompanying drawings, in which:

FIG. 1 is a block diagram of an apparatus according to an embodiment ofthe invention;

FIG. 2 illustrates examples of providing semantic information usinglight;

FIG. 3 is a flow chart illustrating a method of operating an apparatusaccording to an embodiment of the invention;

FIG. 4 illustrates examples of controlling a light emitting portionaccording to embodiments of the invention; and

FIG. 5 illustrates an alternative example of controlling a lightemitting portion according to an embodiment of the invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

FIG. 1 shows an example of an apparatus 100 according to an embodimentof the invention. The apparatus 100 is generally for allowing a user toview themselves and for providing semantic information to the user.Thus, the apparatus 100 comprises a display surface 102 that is forpresenting an image of a user (i.e. the user's reflection) to the userof the apparatus 100. In some embodiments, the display surface 102 is areflective (i.e. mirrored) surface such as that found in a conventionalmirror, and thus the display surface 102 simply presents a reflection ofthe surroundings, including the user when the apparatus 100 is in use.Alternatively, the apparatus 100 can be a so-called ‘virtual mirror’,‘smart mirror’ or ‘digital mirror’ and thus the display surface 102 canbe a display screen that displays an image or video sequence obtained bya camera, image sensor or video sensor associated with the apparatus100. This camera, image sensor or video sensor is not shown in FIG. 1.The camera, image sensor or video sensor can be arranged or orientedwith respect to the display surface 102 such that when a user is infront of the display surface 102, the display surface 102 shows thecaptured image or video sequence of the user. It will be appreciatedthat the captured image or video sequence can be reflected around avertical axis in order for the image or video sequence displayed on thedisplay screen to represent the user's ‘reflection’.

In the case where the display surface 102 is a display screen thatdisplays images or video of the user, it will be appreciated that thedisplay surface 102 preferably presents a real-time or live image orvideo sequence of the user. Thus the image or video is presented on thedisplay screen 102 as quickly as possible after capture by the camera,image sensor or video sensor in order for the apparatus 100 to operateas close to a conventional reflective surface mirror as possible. Ofcourse, it will be appreciated that there will inevitably be a smalldelay between capture of the images or video and display on the displayscreen (e.g. due to image or video processing by the camera, imagesensor or video sensor and the display screen), but this will besufficiently short that the images or video presented on the displayscreen appear to be in real-time or live from the perspective of theuser (and thus the displayed images or video will appear to the user tobe a reflection of the user).

Semantic information is provided to the user using a light emittingportion 104. In particular, the light emitting portion 104 is foremitting light in certain configurations, patterns, colours, etc. inorder to convey information to the user. The light emitting portion 104may be configured or controllable to provide or display words, numerals,symbols, images and/or patterns that are each associated with certainsemantic information. The semantic information can be informationrelating to the performance of a personal care activity by the user orother information that is useful to the user.

The light emitting portion 104 typically comprises a plurality of lightemitting elements. The light emitting elements can be light emittingdiodes, LEDs, organic LEDs, OLEDs, filament-based bulbs, compactfluorescent lamps, CFLs, halogen bulbs, etc. The light emitting elementscan be of different sizes, shapes and/or colours (or have a controllablecolour) and each light emitting element can be controlled individuallyor in combination with one or more other light emitting elements. One ormore, or all, of the light emitting elements can be controlled at anygiven time to provide semantic information to the user.

A control unit 106 is provided that can control the operation of thelight emitting portion 104. In particular, the control unit 106 cancontrol the light emitting portion 104 to provide semantic informationto the user, as required. Where the light emitting portion 104 comprisesa plurality of light emitting elements, the control unit 106 can controlthe light emitting elements, individually and/or in combination in orderto provide the semantic information. It will be appreciated that thecontrol unit 106 can determine the semantic information to be providedto the user using the light emitting portion 104, for example bymonitoring the user using one or more sensors or personal care devices,and/or by obtaining information to be presented to the user (e.g. newsheadlines or a weather report) from an external information source (e.g.the internet).

The control unit 106 can be implemented in numerous ways, with softwareand/or hardware, to perform the required function(s). The control unit106 may comprise one or more microprocessors that may be programmedusing software to perform the required functions. The control unit 106may be implemented as a combination of dedicated hardware to performsome functions and a processor (e.g., one or more programmedmicroprocessors and associated circuitry) to perform other functions.Examples of controller components that may be employed in variousembodiments of the present disclosure include, but are not limited to,conventional microprocessors, application specific integrated circuits(ASICs), and field-programmable gate arrays (FPGAs).

In some embodiments, in order for the face and/or body of the user to bewell-lit when using the apparatus 100 (i.e. to make it easier for theuser to see themselves in the display surface 102), the apparatus 100can further comprise another light source (not shown in FIG. 1) that isprovided for the purpose of evenly lighting the user while they areusing the apparatus 100. As with the light emitting portion 104 above,this light source can comprise one or more LEDs, OLEDs, filament-basedbulbs, CFLs, halogen bulbs, etc.

Alternatively, where only part of the light emitting portion 104 isbeing used to provide semantic information to the user at any giventime, the other part of the light emitting portion 104 can be used toilluminate the user of the apparatus 100. This avoids the need for aseparate light source to be provided for illuminating the user.

In some embodiments, the light emitting portion 104 (and thus the lightemitting elements) is arranged next to, in, on, or partly or completelysurrounding the display surface 102. Thus, for example, the lightemitting portion 104 can form a border for the display surface 102,although those skilled in the art will be aware of other ways in whichthe light emitting portion 104 and the display surface 102 can bearranged. For example, the light emitting portion 104 can be arrangedalong one or more sides of the display surface 104, or arranged toprimarily illuminate a wall or other surface behind the display surface102 so that the user can perceive the semantic information from thelight illuminating the wall or other surface.

In some examples, the semantic information provided by the lightemitting portion can relate to time, for example a counter, timer or aprogress bar relating to the time remaining on a counter for a personalcare activity, such as brushing teeth. Another example of semanticinformation relating to a personal care activity can be an indication ofwhich area(s) of the face have/have not been cleaned or shaved, or whichquadrants of the mouth have/have not been brushed when brushing teeth. Afurther example of semantic information can include an indication ofwhether the pressure applied by the user in brushing their teeth orexfoliating their skin is sufficient (or excessive). A further exampleof semantic information is an indication of whether the position ororientation of a personal care device (e.g. a toothbrush, shaver, hairtrimmer, etc.) is correct. Yet another example of semantic informationthat may be displayed by the light emitting portion 104 includes healthinformation such as body weight, body weight trends. Those skilled inthe art will be aware of many other types of semantic information thatcould be presented to a user using light from a light emitting portion104.

FIG. 2 illustrates four examples of how semantic information can beindicated to a user using light. In FIG. 2, the apparatus 100 is shownwith a circular display surface 102 where the light emitting portion 104extends around the complete periphery of the display surface. It will beappreciated that the display surface 102 can have a different shape tothat shown, and/or the light emitting portion 104 can be arrangeddifferently (e.g. just on part of the sides, or within the periphery ofthe display surface 102).

Thus, in FIG. 2a , the semantic information is provided by emittinglight from a small segment 150 of the light emitting portion 104, andmoving the segment 150 around the display surface 104 over time, asindicated by the arrow 151. This can be used, for example, to indicate atime elapsed or time remaining for a personal care activity. It will beappreciated that the light emitting portion 104 itself does not move,and thus the ‘movement’ of the segment 150 is achieved by selectivelyilluminating different (e.g. successive) parts of the light emittingportion 104 over time. The segment 150 may be provided by one or more ofthe light emitting elements in the light emitting portion 104. In someembodiments the other part 152 of the light emitting portion 104 (i.e.the parts other than segment 150), can be switched off while segment 150is lit, but in other embodiments part 152 can be continuously lit,and/or lit in a different colour or intensity to the segment 150 inorder to illuminate the user.

The illumination shown in FIG. 2b can be used to illustrate timeelapsed. In this example, a progress/timer bar segment 154 is graduallyincreased in size as time elapses. Thus, as time elapses, more parts ofthe light emitting portion 104 are lit to form the progress/timer bar154, as indicated by arrow 155. As with the example in FIG. 2a , theremaining segment 156 of the light emitting portion 104 can be switchedoff until required, or it can be continuously lit and/or lit in adifferent colour or intensity to the bar 154 in order to illuminate theuser.

In FIG. 2c , the semantic information is in the form of a trend/statusindicator 158, 160, where status/trend against a target is indicated byhow much of the trend/status bar 158, 160 is lit. This can be used, forexample, to show whether the user is applying the correct pressure witha toothbrush, where the correct amount of pressure corresponds to bar158 being completely lit, and where an excess amount of pressurecorresponds to bar 160 being lit as well (for example in red to indicatethat the user is pressing too hard). As with the examples in FIGS. 2aand 2b , the remaining segment(s) 162 of the light emitting portion 104can be switched off, or they can be continuously lit and/or lit in adifferent colour or intensity to the bar 158, 160 in order to illuminatethe user.

FIG. 2d shows a form of illumination that can be used to representwhether a user has brushed the teeth in each quadrant of their mouthcorrectly. Thus, the light emitting portion 104 is divided into foursegments 164, 166, 168, 170, that each represent a respective quadrantof the mouth. In this example, the colour of each segment 164, 166, 168,170 is used to indicate whether the corresponding quadrant has beenbrushed correctly. Thus, segment 168 and 170 could be green to indicatethat those quadrants have been brushed correctly, segment 166 could beyellow to indicate that that quadrant has only been partially brushed,and segment 164 can be red to indicate that that quadrant has not beenbrushed at all. The colour of each segment can be controlled and updatedas the user brushes their teeth.

As noted above, a problem exists in that it is desirable for a user tobe evenly lit (either with the ambient light or by the light emittingportion 104 or separate light source) so that the user can optimally seetheir face and/or body in the display surface 102, but using light topresent semantic information to the user can affect the evenness of thelighting on the user (and thus on the image of the user presented by thedisplay surface). For example, providing semantic information byemitting light (particularly non-white light) can cause shadowing orcolour-casting on the user.

It has been recognised that the effect of the light used to convey thesemantic information depends on how close the user is to the lightemitting portion 104 (for example shadowing and/or colour-casting on theuser are more pronounced when the user is closer to the light emittingportion 104 than further away), and therefore it has been found that thecharacteristics of the light used to convey the semantic information canbe adjusted to minimise the effect of the shadowing or colour-casting.Thus, in accordance with the invention, the apparatus 100 is providedwith a sensor 108 for measuring the position of the user relative to thelight emitting portion 104, and the control unit 106 is configured tocontrol the light emitting portion 104 based on the measured position ofthe user relative to the light emitting portion 104 so as to minimisethe effect of the light used to convey the semantic information providedby the light emitting portion 104 on the presented image of the user.

The sensor 108 is configured to measure the position of the userrelative to the light emitting portion 104. In some embodiments, thesensor 108 is a distance sensor that directly measures the distance fromthe light emitting portion 104 to the user (or more generally from thesensor 108 in the apparatus 100 to the user). The sensor 108 can outputa signal representing the measured distance to the control unit 106, oroutput a signal to the control unit 106 that can be processed oranalysed by the control unit 106 to determine the distance. The sensor108 can be any suitable type of distance sensor, for example an IRproximity sensor, ultrasonic distance sensor or a camera or other imagesensor (whose images can be processed, for example by control unit 106,to determine the distance to a user in the image, or to determine adistance associated with a certain autofocus level). Where the apparatus100 is a virtual mirror or digital mirror, the images or video sequencescaptured using the camera or other type of image sensor present in thatapparatus 100 can be analysed to determine the position of the userrelative to the light emitting portion 104, and thus a separate sensor108 is not required.

In some embodiments, the sensor 108 may indirectly measure the distancefrom the light emitting portion 104 to the user by determining when auser is within a threshold distance of the light emitting portion 104(or more generally within a threshold distance of the apparatus 100). Inthese embodiments the sensor 108 can be a proximity sensor (for examplean infrared (IR) proximity sensor), and the sensor 108 can output asignal indicating whether or not the user is within the thresholddistance.

As noted above, the control unit 106 is configured to control the lightemitting portion 104 based on the measured position of the user relativeto the light emitting portion 104. In particular, the control unit 106can control the brightness or intensity of the light that provides thesemantic information; the colour, colour saturation, colour balance orblend of colours of the light that provides the semantic information;the number of light emitting elements used to provide the light for thesemantic information; the focus or light emission angle from the lightemitting elements used to provide the semantic information; or anycombination thereof.

Thus, the control unit 106 can reduce the brightness or intensity of thelight that conveys the semantic information when the user is close tothe light emitting portion 104 compared to when the user is further awayfrom the light emitting portion 104.

In addition or alternatively, the control unit 106 can change thecolour, colour saturation, colour balance or blend of colours of thelight that conveys the semantic information when the user is close tothe light emitting portion 104 so that the light does not cause (orcauses negligible) colour casting on the user compared to the colour,colour saturation, colour balance or blend of colours of the light thatis used when the user is further away from the light emitting portion104 and colour-casting is less likely to occur. For example, when theuser is close to the light emitting portion 104, the control unit 106can control the colour, colour saturation, colour balance or blend ofcolours of the light that conveys the semantic information so that thelight is mixed to a neutral white or grey tone at the point where itmeets the user's face, or is desaturated to such a level that the effectof the colour-casting is negligible to the user.

In addition or alternatively, the control unit 106 can reduce the numberof light emitting elements used to provide the light that conveys thesemantic information when the user is close to the light emittingportion 104 compared to when the user is further away from the lightemitting portion 104. The use of fewer light emitting elements reducesthe total light output by the light emitting portion 104, and thusreduces the amount of additional illumination of the user provided bythe semantic information.

In addition or alternatively, the control unit 106 can change the focusor light emission angle from the light emitting elements used to providethe semantic information (particularly where the light emitting elementsemit a mixture of colours) such that the light is mixed at a shorterdistance from (i.e. closer to) the light emitting portion 104 when theuser is close to the light emitting portion 104 compared to when theuser is further away from the light emitting portion 104.

It will be appreciated that in some embodiments the parameters of thelight emitting portion 104 can be controlled continuously orsemi-continuously based on the measured position or distance (i.e. theparameters are dynamically changing with the measured position ordistance). In other embodiments the parameters of the light emittingportion 104 can be changed in a step-wise manner based on the measuredposition or distance. In further embodiments, the parameters of thelight emitting portion 104 can have two settings, one setting based onthe user being close to the light emitting portion 104 (e.g. when theuser is within a threshold distance of the light emitting portion 104)and the other setting based on the user being sufficiently far from thelight emitting portion 104 (e.g. when the user is more than thethreshold distance from the light emitting portion 104).

Controlling the light emitting portion 104 as outlined above based onthe measured position of the user reduces the shadowing orcolour-casting effect from the light conveying the semantic informationon the user when the user is close to the apparatus 100. Although insome embodiments this effect is achieved at the expense of thebrightness or saturation of the light that conveys the semanticinformation, when the user is not close to the light emitting portion104 brighter and/or more saturated light can still be used to providethe semantic information.

The flow chart in FIG. 3 illustrates a method of operating the apparatus100 according to the invention. In step 1001, the position of a userrelative to a light emitting portion 104 of the apparatus 100 ismeasured. As noted above, the position can be measured using a sensor108 in the apparatus 100. In some embodiments, the position is measuredin terms of the distance of the user from the light emitting portion 104or apparatus 100, or in terms of whether the user is within a thresholddistance of the light emitting portion 104.

When the apparatus 100 is to provide semantic information to the userusing light from a light emitting portion 104 of the apparatus 100, thecontrol unit 106 controls the light emitting portion 104 to provide thesemantic information, and in particular controls the light emittingportion 104 based on the measured position, so as to minimise the effect(e.g. colour-casting and/or shadowing) of the light conveying thesemantic information on the presented image of the user (step 1003).

Thus, in step 1003, the control unit 106 controls the light emittingportion 104 based on the measured position of the user relative to thelight emitting portion 104 to control the brightness or intensity of thelight that provides the semantic information; the colour, coloursaturation, colour balance or blend of colours of the light thatprovides the semantic information; the number of light emitting elementsused to provide the light for the semantic information; and/or the focusor light emission angle from the light emitting elements used to providethe semantic information.

FIG. 4 illustrates various examples of how the light emitting portion104 can be controlled according to embodiments of the invention. Theapparatus 100 illustrated in FIG. 4 is similar to the apparatus 100shown in FIG. 2, and thus comprises a circular display surface 102 wherethe light emitting portion 104 extends around the complete periphery ofthe display surface 102. As with FIG. 2, it will be appreciated that thedisplay surface 102 can have a different shape to that shown, and/or thelight emitting portion 104 can be arranged differently (e.g. just onpart of the sides, or within the periphery of the display surface 102).

FIGS. 4a and 4b show a first example of how the light emitting portion104 can be controlled to minimise the effect of the light conveying thesemantic information on the user. The semantic information in FIGS. 4aand 4b , or rather the manner of indicating the semantic information, issimilar to that shown in FIG. 2a and thus comprises a segment 202 thatis moved around the periphery of the display surface 102 to indicate,for example, a time elapsed or time remaining for a personal careactivity. The remaining segment 204 of the light emitting portion 104 isnot used to convey the semantic information (for example the remainingsegment 204 does not emit light at all or is used to generallyilluminate the user). The segment 202 is formed from a plurality oflight emitting elements.

When the user is close to the light emitting portion 104, the light fromsegment 202 can cause shadowing or colour-casting on the user's face,and this can be visible to the user in the image presented on thedisplay surface 104. Thus, when it is determined that the user is notclose to the light emitting portion 104 (e.g. the measured positionindicates that the user is more than a threshold distance from the lightemitting portion 104), the segment 202 can be as shown in FIG. 4a .However, if the user is close to the light emitting portion 104 (e.g.the measured position indicates that the user is less than a thresholddistance from the light emitting portion 104), the size of the segment202 can be reduced (for example by representing the segment 202 withless light emitting elements than in FIG. 4a ), as shown in FIG. 4 b.

It will be appreciated that the change in the number of light emittingelements forming the segment 202 is not restricted to the moving segment202 type of semantic information shown in FIGS. 4a and 4b , and it canbe applied to any way in which semantic information is conveyed to theuser.

FIGS. 4c and 4d show a second example of how the light emitting portion104 can be controlled to minimise the effect of the light conveying thesemantic information on the user. The semantic information in FIG. 4ccomprises two segments 210 that are positioned on opposite sides of thedisplay surface 102 (although it will be appreciated that theillustrated positions of the segments 210 are not limiting). Theremaining segments 212 of the light emitting portion 104 are not used toconvey the semantic information (for example the remaining segments 212do not emit light at all or are used to generally illuminate the user).The segment 210 can be formed from one or more light emitting elements.In FIG. 4c the user is not close to the light emitting portion 104, andso the segments 210 convey the semantic information using light with adesired colour saturation, colour or colour blend.

When the user is close to the light emitting portion 104, the light fromsegment 210 can cause shadowing or colour-casting on the user's face,and this can be visible to the user in the image presented on thedisplay surface 104. Thus, when it is determined that the user is closeto the light emitting portion 104 (e.g. the measured position indicatesthat the user is less than a threshold distance from the light emittingportion 104), the control unit 106 can control the light emittingportion 104 so that the colour saturation, colour or colour blend of thelight forming the segments 210 causes less shadowing or colour-castingon the user. Thus, as shown in FIG. 4d , the light emitting portion 104can be controlled so that the semantic information is conveyed using asegment 214 (that is smaller than segment 210 in FIG. 4c ) with light ofthe same colour saturation, colour or colour blend as segment 210 inFIG. 4c , and two other small segments 216 are provided either side ofsegment 214 that have a different colour saturation, colour or colourblend to segment 214, so that the combined effect of segments 214 and216 is that the shadowing or colour-casting on the user is reducedcompared to that from segment 210 in FIG. 4 c.

For example the segments 210 can be a bright red colour, and when theuser is closer to the light emitting portion 104, the size of the lightemitting portion that is a bright red colour (segment 214) can bereduced, and neighbouring segments 216 with a less saturated colour(e.g. light pink) can be used to reduce the colour-casting or shadowingon the user.

It will be appreciated that the change in the colour saturation, colouror colour blend of the segment 210 is not restricted to the type ofsemantic information shown in FIGS. 4c and 4d , and it can be applied toany way in which semantic information is conveyed to the user.

FIGS. 4e and 4f show a third example of how the light emitting portion104 can be controlled to minimise the effect of the light conveying thesemantic information on the user. The semantic information in FIGS. 4eand 4f comprises four segments that are individually controllable toconvey semantic information to the user. The type of semanticinformation provided in FIGS. 4e and 4f can be as shown in FIG. 2d ,e.g. with each segment representing whether a quadrant in the user'smouth has been brushed correctly.

Thus, in FIG. 4e , four segments 220 are provided that each display aparticular colour at a particular brightness or intensity level. Theremaining segments 222 of the light emitting portion 104 are not used toconvey the semantic information (for example the remaining segments 222do not emit light at all or are used to generally illuminate the user).Each segment 220 can be formed from one or more light emitting elements.In FIG. 4e the user is not close to the light emitting portion 104, andso the segments 220 convey the semantic information using light with adesired brightness or intensity.

When the user is close to the light emitting portion 104, the light fromsegments 220 can cause shadowing or colour-casting on the user's face,and this can be visible to the user in the image presented on thedisplay surface 104. Thus, when it is determined that the user is closeto the light emitting portion 104 (e.g. the measured position indicatesthat the user is less than a threshold distance from the light emittingportion 104), the control unit 106 can control the light emittingportion 104 so that the brightness or intensity of the light forming thesegments 220 causes less shadowing or colour-casting on the user. Thus,as shown in FIG. 4f , the light emitting portion 104 can be controlledso that the semantic information is conveyed using segments 224 withlight of a lower brightness intensity or colour saturation than used insegment 220 in FIG. 4e (segments 226 are not used to convey semanticinformation, similar to segments 222 in FIG. 4e ). Thus, the lowerbrightness and/or intensity of the segments 224 in FIG. 4f will reducethe shadowing or colour-casting on the user compared to that fromsegments 220 in FIG. 4 e.

It will be appreciated that the change in the brightness intensity orcolour saturation of the segments 220 is not restricted to the type ofsemantic information shown in FIGS. 4e and 4f , and it can be applied toany way in which semantic information is conveyed to the user.

FIGS. 4g and 4h show a fourth example of how the light emitting portion104 can be controlled to minimise the effect of the light conveying thesemantic information on the user. The semantic information in FIGS. 4gand 4h is similar to that shown in FIGS. 4a and 4b and thus comprises asegment 228 that moves around the light emitting portion 104. In thisexample, the remaining segment 230 of the light emitting portion 104 isnot used to convey the semantic information, but is used to generallyilluminate the user.

In this example, the moving segment 228 is formed of a red segment 228r, a green segment 228 g and a blue segment 228 b which on averageproduce light that has the same colour (i.e. white) and intensity as theremaining segment 230. However, when the user moves closer to the lightemitting portion 104, the user can perceive the red, green and bluesegments rather than the ‘averaged’ white light that is intended (andthus red, green and blue light can be cast onto the user), and so whenthe user is close to the light emitting portion 104 (e.g. the measuredposition indicates that the user is less than a threshold distance fromthe light emitting portion 104), the size of the red, green and bluesegments 228 can be reduced as shown by segments 232 r, 232 g and 232 bin FIG. 4h . This change in the size of the segments 232 means that thered, green and blue segments can still create the desired light mix forthe closer distance, and so the user will still perceive the ‘averaged’colour (e.g. white) as intended (and generally at the intended size),and red, green and blue colour casting on the user will be reduced.

As noted above, in some embodiments, the control unit 106 can change thefocus or light emission angle from the light emitting elements used toprovide the semantic information (particularly where the light emittingelements emit a mixture of colours) such that the light is mixed at ashorter distance from (i.e. closer to) the light emitting portion 104when the user is close to the light emitting portion 104 compared towhen the user is further away from the light emitting portion 104. Thisis illustrated in FIG. 5. FIGS. 5a and 5b show three light emittingelements 250 that each have an adjustable optical arrangement 252 (e.g.an adjustable lens or lens arrangement) that can change the focus of thelight emitted by each light emitting element 250 or that can change theemission angle of the light from each light emitting element 250.Adjusting the optical arrangement 252 can change the distance from thelight emitting portion 104 at which light from each light emittingelement 250 is mixed. For example, as shown in FIG. 5a , when the useris not close to the light emitting portion 104 (e.g. the measuredposition indicates that the user is more than a threshold distance fromthe light emitting portion 104), the optical arrangements 252 can beadjusted by the control unit 106 such that the light is mixed furtherfrom the light emitting portion 104 (i.e. the emission angle isnarrower) than when the user is close to the light emitting portion 104(as shown in FIG. 5b ), i.e. where the focus is closer to the lightemitting portion 104 (i.e. the emission angle is wider).

In some embodiments, the control of the light emitting portion 104 basedon the measured position described above can be predetermined and therequired control programmed into the control unit 106 (i.e. for aparticular measured position the light emitting portion 104 is to becontrolled in a certain way). However, in other embodiments, the controlunit 106 can predict a colour casting or shadowing effect of the lightused to convey the semantic information, and use this prediction withthe measured position to determine how to control the light emittingportion 104. For example, in some embodiments the apparatus 100 cancomprise a sensor for detecting an ambient light level or contributionof ambient light (for example a light sensor or a camera or imagesensor), and the control unit 106 can control the light emitting portion104 based on the measured position and the measured ambient light levelor detected ambient light contribution. For example, light used toprovide semantic information to the user is more likely to causeshadowing or colour casting when the ambient light level is low thanwhen the ambient light level is normal or higher. Likewise, light usedto provide semantic information to the user is more likely to causeshadowing or colour casting on parts of the user where the contributionto the lighting of the user due to ambient lighting is low. Thus, if itis determined that the ambient light level or ambient light contributionis low, then the control unit 106 may make a larger adjustment to theparameter(s) of the light emitting portion 104 that determines thecharacteristics of the semantic information when the user is close tothe light emitting portion 104 than when the ambient light level orambient light contribution is high or normal. In some embodiments, wherethe ambient light level sensor is a camera or image sensor, the controlunit 106 can determine the ambient light level or ambient lightcontribution from a comparison of images in which the user isilluminated with light from the apparatus 100 (including the lightconveying the semantic information) and images in which the user is notilluminated (or not evenly illuminated) with light from the apparatus100. In particular the control unit 106 can ‘subtract’ the images inwhich the user is not illuminated from images in which the user isilluminated to determine the ambient light level or ambient lightcontribution.

In some embodiments, the control of the light emitting portion 104 canbe based on the measured position of the user relative to the lightemitting portion 104 and on an analysis or observation of the lightconveying the semantic information on the user. In these embodiments,the apparatus 100 can comprise a camera or other image sensor (which canbe the same as the camera or image sensor described in the aboveembodiments) that obtains one or more images or a video sequence of theuser as the semantic information is conveyed to the user using the lightemitting portion 104, and the control unit 106 analyses the obtainedimages or video sequence to determine the effect of the light conveyingthe semantic information on the user and/or to determine the evenness oflighting on the user (e.g. determine whether or which parts of the userare more shadowed than others). The control unit 106 can then determinean adjustment to the light emitting portion 104 to minimise the effectof the light conveying the semantic information on the user. Forexample, if the semantic information is being conveyed using red light,the control unit 106 can analyse the images or video sequence todetermine if the red light is casting onto the user, and if so, thecontrol unit 106 can control the light emitting portion 104 to reducethe brightness, intensity or colour saturation, or change the colour orcolour blend to minimise the light casting on the user. It will beappreciated that in these embodiments the control unit 106 can firstlyeffect an adjustment or control of the light emitting portion 104 basedon the measured position of the user relative to the light emittingportion 104 and then analyse images or a video sequence of the user todetermine if further adjustment or control is necessary. Alternatively,it will be appreciated that in these embodiments the control unit 106can control the light emitting portion 104 to present the semanticinformation to the user and effect a single adjustment or control of thelight emitting portion 104 based on the measured position of the userrelative to the light emitting portion 104 and an analysis of images ora video sequence of the user.

Thus, there is provided an improved apparatus and method of operating anapparatus that minimises the effect of light conveying semanticinformation on a presented image of a user.

Variations to the disclosed embodiments can be understood and effectedby those skilled in the art in practicing the claimed invention, from astudy of the drawings, the disclosure and the appended claims. In theclaims, the word “comprising” does not exclude other elements or steps,and the indefinite article “a” or “an” does not exclude a plurality. Asingle processor or other unit may fulfil the functions of several itemsrecited in the claims. The mere fact that certain measures are recitedin mutually different dependent claims does not indicate that acombination of these measures cannot be used to advantage. A computerprogram may be stored/distributed on a suitable medium, such as anoptical storage medium or a solid-state medium supplied together with oras part of other hardware, but may also be distributed in other forms,such as via the Internet or other wired or wireless telecommunicationsystems.

Any reference signs in the claims should not be construed as limitingthe scope.

1. An apparatus comprising: a display surface for presenting a real-timeimage of a user; a light emitting portion for using light to conveysemantic information to the user; a sensor for measuring the position ofthe user relative to the light emitting portion; and a control unitconfigured to control the light emitting portion based on the measuredposition of the user relative to the light emitting portion so as tominimise a shadowing and/or colour-casting effect of the light conveyingthe semantic information provided by the light emitting portion on theuser and the presented real-time image of the user.
 2. The apparatusaccording to claim 1, wherein the control unit is configured to controlthe light emitting portion by adjusting the brightness or intensity ofthe light used to convey the semantic information based on the measuredposition of the user relative to the light emitting portion.
 3. Theapparatus according to claim 2, wherein the control unit is configuredto control the brightness or intensity of the light emitting portionsuch that the brightness or intensity of the light conveying thesemantic information is lower when the user is closer to the lightemitting portion than when the user is further from the light emittingportion.
 4. The apparatus according to claim 1, wherein the control unitis configured to control the light emitting portion by adjusting thecolour, colour saturation and/or colour blend of the light used toprovide the semantic information based on the measured position of theuser relative to the light emitting portion.
 5. The apparatus accordingto claim 1, wherein the light emitting portion comprises a plurality oflight emitting elements that can be used to convey the semanticinformation, wherein the control unit is configured to control thenumber of light emitting elements used to convey the semanticinformation based on the measured position of the user relative to thelight emitting portion.
 6. The apparatus according to claim 1, whereinthe light emitting portion comprises a plurality of light emittingelements that can be used to convey the semantic information, andwherein the control unit is configured to control a light emission angleor focus of the light emitted by the light emitting elements based onthe measured position of the user relative to the light emittingportion.
 7. The apparatus according to claim 1, the apparatus furthercomprising: a sensor for obtaining one or more images or a videosequence of a user of the apparatus; and wherein the control unit isfurther configured to: analyse the obtained one or more images or videosequence to determine a shadowing and/or colour-casting effect of thelight conveying the semantic information on the user and/or to determinethe evenness of lighting on the user; and control the light emittingportion based on the measured position of the user relative to the lightemitting portion and the determined effect of the light and/ordetermined evenness of the lighting so as to minimise the shadowingand/or colour-casting effect of the light conveying the semanticinformation provided by the light emitting portion on the user and thepresented real-time image of the user.
 8. The apparatus according toclaim 1, the apparatus further comprising: a sensor for detecting anambient light level or contribution of ambient light; and wherein thecontrol unit is configured to control the light emitting portion so asto minimise the shadowing and/or colour-casting effect of the semanticinformation provided by the light emitting portion on the user and thepresented real-time image of the user based on the ambient light and themeasured position of the user relative to the light emitting portion. 9.A method of operating an apparatus to convey semantic information to auser of the apparatus, the apparatus comprising a display surface fordisplaying a real-time image of the user, the method comprising:measuring the position of a user relative to a light emitting portion ofthe apparatus; and controlling the light emitting portion to emit lightto convey the semantic information to the user, wherein the lightemitting portion is controlled based on the measured position so as tominimise a shadowing and/or colour-casting effect of the light conveyingthe semantic information on the user and a real-time image of the userpresented on the display surface.
 10. The method as claimed in claim 9,wherein the step of controlling comprises controlling the light emittingportion by adjusting the brightness or intensity of the light used toconvey the semantic information based on the measured position of theuser relative to the light emitting portion.
 11. The method as claimedin claim 9, wherein the step of controlling comprises controlling thelight emitting portion by adjusting the colour, colour saturation and/orcolour blend of the light used to provide the semantic information basedon the measured position of the user relative to the light emittingportion.
 12. The method as claimed in claim 9, wherein the lightemitting portion comprises a plurality of light emitting elements thatcan be used to convey the semantic information, and wherein the step ofcontrolling comprises controlling the number of light emitting elementsused to convey the semantic information based on the measured positionof the user relative to the light emitting portion.
 13. The method asclaimed in claim 9, wherein the light emitting portion comprises aplurality of light emitting elements that can be used to convey thesemantic information, and the step of controlling comprises controllinga light emission angle or focus of the light emitted by the lightemitting elements based on the measured position of the user relative tothe light emitting portion.
 14. The method as claimed in claim 9,wherein the method further comprises the steps of: obtaining one or moreimages or a video sequence of a user of the apparatus; analysing theobtained one or more images or video sequence to determine a shadowingand/or colour-casting effect of the light conveying the semanticinformation on the user and/or to determine the evenness of lighting onthe user; and wherein the step of controlling comprises controlling thelight emitting portion based on the measured position of the userrelative to the light emitting portion and the determined effect of thelight and/or determined evenness of the lighting so as to minimise theshadowing and/or colour-casting effect of the light conveying thesemantic information provided by the light emitting portion on the userand the presented real-time image of the user.
 15. The method as claimedin claim 9, wherein the method further comprises the step of: detectingan ambient light level or contribution of ambient light; and wherein thestep of controlling comprises controlling the light emitting portion soas to minimise the shadowing and/or colour-casting effect of thesemantic information provided by the light emitting portion on the userand the presented real-time image of the user based on the ambient lightand the measured position of the user relative to the light emittingportion.